Development and testing of a portfolio evaluation scoring tool.

This study focused on development of a portfolio evaluation tool to guide the assignment of valid and reliable scores. Tool development was facilitated by a literature review, guidance of a faculty committee, and validation by content experts. Testing involved a faculty team that evaluated 60 portfolios. Calculation of interrater reliability and a paired-samples t test were used to judge effectiveness. Interrater reliability was 0.78 for overall scores, 0.81 for the seven program outcomes criteria scores, and more than 0.65 for scores assigned by 11 of 13 pairs of raters. There were no significant differences between raters' scores in 10 of 13 pairs. The portfolio evaluation tool demonstrated high reliability and should be tested by other schools using portfolio evaluation.